Unmarked crosswalks for autonomous vehicles

ABSTRACT

Aspects of the disclosure provide for controlling an autonomous vehicle having an autonomous driving mode. For instance, sensor data generated by a perception system of a vehicle may be received. The sensor data identifying objects in an environment of the vehicle. A trajectory corresponding to a future path of the vehicle may be received. That the vehicle is approaching an unmarked crosswalk situation may be determined based on the trajectory and the sensor data. In response to the determination, an area for an unmarked crosswalk is identified. The vehicle may be controlled in the autonomous driving mode based on the determination and the area.

BACKGROUND

Autonomous vehicles, such as vehicles which do not require a humandriver when operating in an autonomous driving mode, may be used to aidin the transport of passengers or items from one location to another. Insome instances, these vehicles may need to change from the autonomousdriving mode, where the vehicle's computing devices control thesteering, acceleration and deceleration of the vehicle to asemi-autonomous driving mode, where there is shared control between thevehicle's computing devices and a driver of the vehicle (e.g. the drivermay control the steering of the vehicle via a steering input while thevehicle's computing devices control the acceleration and deceleration ofthe vehicle), or to a manual driving mode where a driver controls thesteering, acceleration and deceleration of the vehicle via steering,acceleration, and deceleration inputs.

These vehicles may also be equipped with systems for responding tocrosswalks. Crosswalks may be identified in map information, via visualdetection methods using sensor data generated by a perception system ofan autonomous vehicle, or a combination of both. Once a crosswalk isidentified, the vehicle may engage certain crosswalk behaviors whenapproaching crosswalks such as occlusion reasoning, more cautiousbehaviors, assuming certain objects may be pedestrians even if aconfidence in such determinations would otherwise be considered too low,etc. Such behaviors may enable the vehicle to safely maneuver throughthe crosswalk.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of an example vehicle in accordance withan exemplary embodiment.

FIG. 2 is an example diagram of a vehicle in accordance with aspects ofthe disclosure.

FIG. 3 is an example flow diagram in accordance with aspects of thedisclosure.

FIG. 4 is an example bird's eye view of a section of roadway 400 inaccordance with aspects of the disclosure.

FIGS. 5A-5C is an example situation and areas of an unmarked crosswalkin accordance with aspects of the disclosure.

FIGS. 6A-6B is an example situation and area of unmarked crosswalks inaccordance with aspects of the disclosure.

FIGS. 7A-7C is an example situation and areas of unmarked crosswalks inaccordance with aspects of the disclosure.

FIGS. 8A-8B is an example situation and area of an unmarked crosswalk inaccordance with aspects of the disclosure.

FIGS. 9A-9B is an example situation and area of an unmarked crosswalk inaccordance with aspects of the disclosure.

FIGS. 10A-10B is an example situation and area of an unmarked crosswalkin accordance with aspects of the disclosure.

SUMMARY

One aspect of the disclosure provides a method for controlling anautonomous vehicle having an autonomous driving mode. The methodincludes receiving, by one or more processors of the vehicle, sensordata generated by a perception system of a vehicle, the sensor dataidentifying objects in an environment of the vehicle; receiving, by theone or more processors of the vehicle, a trajectory corresponding to afuture path of the vehicle; determining, by the one or more processorsof the vehicle, that the vehicle is approaching an unmarked crosswalksituation based on the trajectory and the sensor data; in response tothe determination, identifying, by the one or more processors of thevehicle, an area for an unmarked crosswalk; and controlling, by the oneor more processors of the vehicle, the vehicle in the autonomous drivingmode based on the determination and the area.

In one example, the unmarked crosswalk situation corresponds to asituation in which there is no marked crosswalk in the environment. Inthis example, the unmarked crosswalk situation corresponds to asituation in which there is no marked crosswalk in any pre-stored mapinformation used to control the vehicle in the autonomous driving mode.In another example, determining that the vehicle is approaching anunmarked crosswalk situation includes determining whether theenvironment includes a school zone. In another example, determining thatthe vehicle is approaching an unmarked crosswalk situation includesdetermining whether the environment includes a school bus. In thisexample, identifying the area is based on a location of the school bus.In another example, determining that the vehicle is approaching anunmarked crosswalk situation includes determining whether theenvironment includes a construction zone. In this example, identifyingthe area is based on a polygon of the construction zone. In anotherexample, determining that the vehicle is approaching an unmarkedcrosswalk situation includes determining whether the environmentincludes one or more persons directing traffic. In this example,identifying the area is based on a location of the one or more personsdirecting traffic. In another example, determining that the vehicle isapproaching an unmarked crosswalk situation includes determining whetherthe environment includes a traffic accident. In this example,identifying the area is based on a polygon for an accident zone of theaccident. In another example, determining that the vehicle isapproaching an unmarked crosswalk situation includes determining whetherthe environment includes one or more emergency vehicles. In thisexample, identifying the area is based on a location of a first of theplurality of emergency vehicles. In addition, the method also includesidentifying a second area based on a location of a second of theplurality of emergency vehicles, and wherein controlling the vehicle isfurther based on the second area. In another example, controlling thevehicle includes giving a pedestrian in the area precedence for crossinga road with respect to the vehicle. In another example, controlling thevehicle includes yielding to a pedestrian on a side of the road outsideof the area. In another example, controlling the vehicle includes usingocclusion reasoning for the area in order to predict one or morepotentially occluded pedestrians. In another example, controlling thevehicle includes allowing the vehicle's computing devices to increaseallocation of computing resources for detecting and responding topedestrians as the vehicle approaches the area.

DETAILED DESCRIPTION Overview

The technology relates to engaging certain behaviors for autonomousvehicles. As noted above, crosswalks may be identified in mapinformation, via visual detection methods using sensor data generated bya perception system of an autonomous vehicle, or a combination of both.Once a crosswalk is identified, the vehicle may engage certain crosswalkbehaviors when approaching crosswalks such as occlusion reasoning, morecautious behaviors, assuming certain objects may be pedestrians even ifa confidence in such determinations would otherwise be considered toolow, etc. Such behaviors may enable the vehicle to safely maneuverthrough the crosswalk.

However, such behaviors may be useful in many other situations includingthose where pedestrians are likely to cross a road on which the vehicleis traveling even in situations where there is no actual crosswalk inthe map information or on the road. In some such situations, thepedestrian may even have precedence, but without being able to engagethe aforementioned crosswalk behaviors. To address this, the vehicle maybe programmed to detect such situations and identify areas as unmarked,contextual, hypothetical, or virtual crosswalks (even though there is noactual crosswalk) in order to engage the crosswalk behaviors.

As the vehicle drives through its environment, an unmarked crosswalkdetection software module may process information from the mapinformation as well as sensor data generated by the perception system todetermine whether the vehicle is in or approaching an unmarked crosswalksituation. For instance, the detection module may determine whether thevehicle's trajectory passes through or nearby a location in a way thatsatisfies one or more predetermined characteristics. Thesecharacteristics may relate to the context in which the vehicle isdriving and/or the current time (e.g. date and/or time). In order tomake the detector as effective as possible, the detection module may berun continuously rather than only at specific locations or times.

Once an unmarked crosswalk situation is identified, the vehicle'scomputing devices may identify an area corresponding to an unmarkedcrosswalk. This area may be determined based on the satisfied one ormore predetermined characteristics and/or the type of the unmarkedcrosswalk situation. In other words, these details may be used toidentify reference points where the crosswalk should start and end, andthen a polygon may be drawn from these points to the locations of roadedges or beyond. Alternatively, a polygon may be drawn using thereference points or around another polygon as discussed further below.Once an area for an unmarked crosswalk is determined, certain behaviorsfor crosswalks may be engaged or otherwise “turned on” to enable thevehicle to react to the unmarked crosswalk as if it were an actualcrosswalk.

The features described herein may enable an autonomous vehicle toidentify situations in which pedestrians may be likely to be crossing,attempting to cross, or simply walking in a road. As such, the vehicleis able to automatically engage certain behaviors useful for crosswalksin instances where such behaviors would be safer, more helpful, andappropriate even if not legally required. Overall, the featuresdescribed herein may greatly improve safety in situations wherepedestrians could be crossing a road outside of a crosswalk.

Example Systems

As shown in FIG. 1, a vehicle 100 in accordance with one aspect of thedisclosure includes various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, buses, recreational vehicles,etc. The vehicle may have one or more computing devices, such ascomputing device 110 containing one or more processors 120, memory 130and other components typically present in general purpose computingdevices.

The memory 130 stores information accessible by the one or moreprocessors 120, including instructions 132 and data 134 that may beexecuted or otherwise used by the processor 120. The memory 130 may beof any type capable of storing information accessible by the processor,including a computing device-readable medium, or other medium thatstores data that may be read with the aid of an electronic device, suchas a hard-drive, memory card, ROM, RAM, DVD or other optical disks, aswell as other write-capable and read-only memories. Systems and methodsmay include different combinations of the foregoing, whereby differentportions of the instructions and data are stored on different types ofmedia.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computingdevice code on the computing device-readable medium. In that regard, theterms “instructions” and “programs” may be used interchangeably herein.The instructions may be stored in object code format for directprocessing by the processor, or in any other computing device languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. For instance, although the claimedsubject matter is not limited by any particular data structure, the datamay be stored in computing device registers, in a relational database asa table having a plurality of different fields and records, XMLdocuments or flat files. The data may also be formatted in any computingdevice-readable format.

The one or more processor 120 may be any conventional processors, suchas commercially available CPUs or GPUs. Alternatively, the one or moreprocessors may be a dedicated device such as an ASIC or otherhardware-based processor. Although FIG. 1 functionally illustrates theprocessor, memory, and other elements of computing device 110 as beingwithin the same block, it will be understood by those of ordinary skillin the art that the processor, computing device, or memory may actuallyinclude multiple processors, computing devices, or memories that may ormay not be stored within the same physical housing. For example, memorymay be a hard drive or other storage media located in a housingdifferent from that of computing device 110. Accordingly, references toa processor or computing device will be understood to include referencesto a collection of processors or computing devices or memories that mayor may not operate in parallel.

The computing devices 110 may also be connected to one or more speakers112 as well as one or more user inputs 114. The speakers may enable thecomputing devices to provide audible messages and information, such asthe alerts described herein, to occupants of the vehicle, including adriver. In some instances, the computing devices may be connected to oneor more vibration devices configured to vibrate based on a signal fromthe computing devices in order to provide haptic feedback to the driverand/or any other occupants of the vehicle. As an example, a vibrationdevice may consist of a vibration motor or one or more linear resonantactuators placed either below or behind one or more occupants of thevehicle, such as embedded into one or more seats of the vehicle.

The user input may include a button, touchscreen, or other devices thatmay enable an occupant of the vehicle, such as a driver, to provideinput to the computing devices 110 as described herein. As an example,the button or an option on the touchscreen may be specifically designedto cause a transition from the autonomous driving mode to the manualdriving mode or the semi autonomous driving mode.

In one aspect the computing devices 110 may be part of an autonomouscontrol system capable of communicating with various components of thevehicle in order to control the vehicle in an autonomous driving mode.For example, returning to FIG. 1, the computing devices 110 may be incommunication with various systems of vehicle 100, such as decelerationsystem 160, acceleration system 162, steering system 164, routing system166, planning system 168, positioning system 170, and perception system172 in order to control the movement, speed, etc. of vehicle 100 inaccordance with the instructions 132 of memory 130 in the autonomousdriving mode.

As an example, computing devices 110 may interact with decelerationsystem 160 and acceleration system 162 in order to control the speed ofthe vehicle. Similarly, steering system 164 may be used by computingdevices 110 in order to control the direction of vehicle 100. Forexample, if vehicle 100 is configured for use on a road, such as a caror truck, the steering system may include components to control theangle of wheels to turn the vehicle.

Planning system 168 may be used by computing devices 110 in order todetermine and follow a route generated by a routing system 166 to alocation. For instance, the routing system 166 may use map informationto determine a route from a current location of the vehicle to a dropoff location. The planning system 168 may periodically generatetrajectories, or short-term plans for controlling the vehicle for someperiod of time into the future, in order to follow the route (a currentroute of the vehicle) to the destination. In this regard, the planningsystem 168, routing system 166, and/or data 134 may store detailed mapinformation, e.g., highly detailed maps identifying the shape andelevation of roadways, lane lines, intersections, crosswalks, speedlimits, traffic signals, buildings, signs, real time trafficinformation, vegetation, or other such objects and information. Inaddition, the map information may identify area types such asconstructions zones, school zones, residential areas, parking lots, etc.

The map information may include one or more roadgraphs or graph networksof information such as roads, lanes, intersections, and the connectionsbetween these features which may be represented by road segments. Eachfeature may be stored as graph data and may be associated withinformation such as a geographic location and whether or not it islinked to other related features, for example, a stop sign may be linkedto a road and an intersection, etc. In some examples, the associateddata may include grid-based indices of a roadgraph to allow forefficient lookup of certain roadgraph features.

Positioning system 170 may be used by computing devices 110 in order todetermine the vehicle's relative or absolute position on a map or on theearth. For example, the positioning system 170 may include a GPSreceiver to determine the device's latitude, longitude and/or altitudeposition. Other location systems such as laser-based localizationsystems, inertial-aided GPS, or camera-based localization may also beused to identify the location of the vehicle. The location of thevehicle may include an absolute geographical location, such as latitude,longitude, and altitude as well as relative location information, suchas location relative to other cars immediately around it which can oftenbe determined with less noise that absolute geographical location.

The positioning system 170 may also include other devices incommunication with the computing devices of the computing devices 110,such as an accelerometer, gyroscope or another direction/speed detectiondevice to determine the direction and speed of the vehicle or changesthereto. By way of example only, an acceleration device may determineits pitch, yaw or roll (or changes thereto) relative to the direction ofgravity or a plane perpendicular thereto. The device may also trackincreases or decreases in speed and the direction of such changes. Thedevice's provision of location and orientation data as set forth hereinmay be provided automatically to the computing device 110, othercomputing devices and combinations of the foregoing.

The perception system 172 also includes one or more components fordetecting objects external to the vehicle such as other vehicles,obstacles in the roadway, traffic signals, signs, trees, etc. Forexample, the perception system 172 may include lasers, sonar, radar,cameras and/or any other detection devices that record data which may beprocessed by the computing devices of the computing devices 110. In thecase where the vehicle is a passenger vehicle such as a minivan, theminivan may include a laser or other sensors mounted on the roof orother convenient location. For instance, FIG. 2 is an example externalview of vehicle 100. In this example, roof-top housing 210 and domehousing 212 may include a LIDAR sensor as well as various cameras andradar units. In addition, housing 220 located at the front end ofvehicle 100 and housings 230, 232 on the driver's and passenger's sidesof the vehicle may each store a LIDAR sensor. For example, housing 230is located in front of driver door 260. Vehicle 100 also includeshousings 240, 242 for radar units and/or cameras also located on theroof of vehicle 100. Additional radar units and cameras (not shown) maybe located at the front and rear ends of vehicle 100 and/or on otherpositions along the roof or roof-top housing 210.

The computing devices 110 may be capable of communicating with variouscomponents of the vehicle in order to control the movement of vehicle100 according to primary vehicle control code of memory of the computingdevices 110. For example, returning to FIG. 1, the computing devices 110may include various computing devices in communication with varioussystems of vehicle 100, such as deceleration system 160, accelerationsystem 162, steering system 164, routing system 166, planning system168, positioning system 170, perception system 172, and power system 174(i.e. the vehicle's engine or motor) in order to control the movement,speed, etc. of vehicle 100 in accordance with the instructions 132 ofmemory 130.

The various systems of the vehicle may function using autonomous vehiclecontrol software in order to determine how to and to control thevehicle. As an example, a perception system software module of theperception system 172 may use sensor data generated by one or moresensors of an autonomous vehicle, such as cameras, LIDAR sensors, radarunits, sonar units, etc., to detect and identify objects and theirfeatures. These features may include location, type, heading,orientation, speed, acceleration, change in acceleration, size, shape,etc. In some instances, features may be input into a behavior predictionsystem software module which uses various behavior models based onobject type to output a predicted future behavior for a detected object.

In other instances, the features may be put into one or more detectionsystem software modules, such as a traffic light detection systemsoftware module configured to detect the states of known trafficsignals, a school bus detection system software module configured todetect school busses, construction zone detection system software moduleconfigured to detect construction zones, a detection system softwaremodule configured to detect one or more persons (e.g. pedestrians)directing traffic, a traffic accident detection system software moduleconfigured to detect a traffic accident, an emergency vehicle detectionsystem configured to detect emergency vehicles, etc. Each of thesedetection system software modules may input sensor data generated by theperception system 172 and/or one or more sensors (and in some instances,map information for an area around the vehicle) into various modelswhich may output a likelihood of a certain traffic light state, alikelihood of an object being a school bus, an area of a constructionzone, a likelihood of an object being a person directing traffic, anarea of a traffic accident, a likelihood of an object being an emergencyvehicle, etc., respectively.

Detected objects, predicted future behaviors, various likelihoods fromdetection system software modules, the map information identifying thevehicle's environment, position information from the positioning system170 identifying the location and orientation of the vehicle, adestination for the vehicle as well as feedback from various othersystems of the vehicle may be input into a planning system softwaremodule of the planning system 168. The planning system may use thisinput to generate trajectories for the vehicle to follow for some briefperiod of time into the future based on a current route of the vehiclegenerated by a routing module of the routing system 166. A controlsystem software module of the computing devices 110 may be configured tocontrol movement of the vehicle, for instance by controlling braking,acceleration and steering of the vehicle, in order to follow atrajectory.

The computing devices 110 may control the vehicle in an autonomousdriving mode by controlling various components. For instance, by way ofexample, the computing devices 110 may navigate the vehicle to adestination location completely autonomously using data from thedetailed map information and planning system 168. The computing devices110 may use the positioning system 170 to determine the vehicle'slocation and perception system 172 to detect and respond to objects whenneeded to reach the location safely. Again, in order to do so, computingdevice 110 may generate trajectories and cause the vehicle to followthese trajectories, for instance, by causing the vehicle to accelerate(e.g., by supplying fuel or other energy to the engine or power system174 by acceleration system 162), decelerate (e.g., by decreasing thefuel supplied to the engine or power system 174, changing gears, and/orby applying brakes by deceleration system 160), change direction (e.g.,by turning the front or rear wheels of vehicle 100 by steering system164), and signal such changes (e.g. by using turn signals). Thus, theacceleration system 162 and deceleration system 160 may be a part of adrivetrain that includes various components between an engine of thevehicle and the wheels of the vehicle. Again, by controlling thesesystems, computing devices 110 may also control the drivetrain of thevehicle in order to maneuver the vehicle autonomously.

Example Methods

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

FIG. 3 provides an example flow diagram 300 for controlling a vehicle inan autonomous driving mode which may be performed by one or moreprocessors of computing devices, such as one or more processors 120 ofcomputing devices 110. For instance, as shown in block 310, sensor datagenerated by a perception system of a vehicle is received. This sensordata identifies objects in an environment of the vehicle. For example,as noted above, the perception system 172 may use the various sensors ofthe vehicle to generate sensor data. The perception system 172 may thensend this sensor data to the computing devices 110.

At block 320, a trajectory corresponding to a future path of the vehicleis received. For instance, as noted above, the planner system 168 mayuse the sensor data, map information, as well as other data to generatea trajectory for the vehicle to follow in order to enable the vehicle toprogress along a route (generated by the routing system 166) in order toreach a destination. The planner system 168 may then send thistrajectory to the computing devices 110.

At block 330, that the vehicle is approaching an unmarked crosswalksituation is determined based on the trajectory and the sensor data. Forinstance, as noted above, as the vehicle drives through its environment,an unmarked crosswalk detection software module of the memory 130 mayprocess information from the map information described above as well assensor data generated by the perception system 172 to determine whetherthe vehicle is in or approaching an unmarked crosswalk situation. Inorder to make the detector as effective as possible, the unmarkedcrosswalk detection software module may be run continuously rather thanonly at specific locations or times. The detection module may determinewhether the vehicle's trajectory passes through or nearby a location ina way that satisfies one or more predetermined characteristics stored inthe memory 130.

These characteristics may relate to the context in which the vehicle isdriving and/or the current time (e.g. date and/or time). For example,the detection model may determine whether the trajectory will cause thevehicle to pass through an intersection without a marked crosswalk, passnearby a school vehicle (e.g. a school van), pass nearby one or morepersons directing traffic, pass through a construction zone, passthrough an area where there are one or more emergency vehicles nearby,pass nearby a traffic accident (e.g. two vehicles colliding or a vehiclecolliding with another object), pass through a school zone on aparticular date or time of day or when school zone sign lights areflashing, a residential area on a particular date or time of day, largerparking lots (e.g. shopping centers, hospitals, parks, etc.), or anyother situations in which pedestrians are likely to be crossing a roadoutside of a crosswalk. As noted above, many of these features, such asintersections, construction zones, school zones, residential areas,parking lots, etc. may be identified in the map information. Thosefeatures and others, including emergency vehicles, traffic accidents,etc. may also be detected in real time using detection software modulesas described above.

FIG. 4 is an example bird's eye view of a section of roadway 400 whichwill be used for demonstration purposes. The roadway 400 includes anintersection 402 as well as fog lines 410, 412, 414 which define theedges of a roadway including lanes 430, 432 divided by lane lines 420,422, 424, 426. In this example, lane 430 meets up with a side streetincluding lanes 434, 436. FIGS. 5A-10B provide example situations inwhich the vehicle 100 is approaching an unmarked crosswalk situation inthe area of roadway 400. In each of these examples, there is nocrosswalk identified in the map information for the area around thevehicle (e.g. what is depicted in each of three examples).

FIG. 5A provides an example situation 500 in which vehicle 100 isdriving in lane 430 and approaching a stopped school bus 510 in lane432. The school bus may be identified, for instance, by the school busdetection software module described above, and the features of theschool bus, such as that the school bus is stopped may be determined bythe perception system 172. In this example, the vehicle's trajectory 512will pass by the school bus 510. Thus, the computing devices 110 maydetermine that the vehicle 100 is approaching an unmarked crosswalksituation.

FIG. 6A provides an example situation 600 in which vehicle 100 isdriving in lane 430 approaching a construction zone 610, for instance,identified by construction zone detection software module describedabove or in the map information. The constructions zone 610 includesconstruction zone objects 620 (e.g. cones), 622 (e.g. a constructionvehicle). In this example, the vehicle's trajectory 612 will passthrough the construction zone 610. Thus, the computing devices 110 maydetermine that the vehicle 100 is approaching an unmarked crosswalksituation.

FIG. 7A provides an example situation 700 in which vehicle 100 isdriving in lane 430 and approaching a person directing traffic 710, forinstance, identified by the detection software module for detectingpersons directing traffic described above. In this example, thevehicle's trajectory 712 will pass by the persons directing traffic 710.Thus, the computing devices 110 may determine that the vehicle 100 isapproaching an unmarked crosswalk situation.

FIG. 8A provides an example situation 800 in which vehicle 100 isdriving in lane 430 and approaching a traffic accident 810, forinstance, identified by the traffic accident detection software moduledescribed above. In this example, the vehicle's trajectory 812 will passby the traffic accident 810. Thus, the computing devices 110 maydetermine that the vehicle is approaching an unmarked crosswalksituation.

FIG. 9A provides an example situation 600 in which vehicle 100 is inlane 430 and approaching an emergency vehicle 910, for instance,identified by the emergency vehicle detection software module describedabove. In this example, the vehicle's trajectory 912 will pass by theemergency vehicle 910. Thus, the computing devices 110 may determinethat the vehicle is approaching an unmarked crosswalk situation.

FIG. 10A provides an example situation 1000 in which vehicle 100 is in aresidential area 1010, for instance, identified in the map information.In this example, the vehicle's trajectory 1012 will pass through theresidential area 1010 including lanes 434, 436. Thus, the computingdevices 110 may determine that the vehicle is approaching an unmarkedcrosswalk situation.

Returning to FIG. 3, at block 340, in response to the determination, anarea for an unmarked crosswalk is identified. For instance, once anunmarked crosswalk situation is determined, the vehicle's computingdevices may identify an area corresponding to an unmarked crosswalk.This area may be determined based on the satisfied one or morepredetermined characteristics and/or the type of the unmarked crosswalksituation. In other words, these details may be used to identifyreference points where the crosswalk should start and end, and then apolygon may be drawn from these points to the locations of road edges orbeyond. Alternatively, a polygon may be drawn using the reference pointsor around another polygon as discussed further below.

For instance, in the presence of active school buses or within schoolzones, children may be expected to cross the road in the vicinity of anactive stopped school bus, or at a school zone even if there is not amapped or painted crosswalk. As such, the area for the unmarkedcrosswalk may be identified using points corresponding to the front andback of the school bus as a reference for drawing a larger rectangularregion around the school bus and across the road. In this regard, therectangle may include some additional buffer area or distance in frontof and behind the school bus, such as an additional 3-meter distance ormore or less. The term “across the width of the road” as used herein mayinclude not only lanes of the road, but also beyond the edges of theroadway (e.g. a shoulder area), and even some small distance beyond theshoulder area in some instances. Alternatively, rather than a singlecrosswalk, a pair of areas for unmarked crosswalks may be determined,one in front of the bus and one behind.

Returning to the example of FIG. 5A, as shown in FIG. 5B, an area 520for an unmarked crosswalk may be determined by using corners on thefront and back of the school bus 510 (or the corners of a bounding boxrepresenting the volume of space occupied by the school bus) as areference for drawing a larger rectangular region around the school busand across the width of both lanes 430 and 432. Arrows 522, 524represent the additional buffer distance. In this example, the area 520extends beyond the edges of the roadway defined by the fog lines 412,414.

Alternatively, as shown in FIG. 5C, areas 520A and 520B for a pair ofunmarked crosswalks may be determined by using corners on the front andback of the school bus 510 (or the corners of a bounding boxrepresenting the volume of space occupied by the school bus) as areference for drawing a rectangular region around the front and rear ofthe school bus and across the width of both lanes 430 and 432. Arrows522A, 522B, 524A, 524B represent the additional forward (A) and reverse(B) buffer distances from the front and back corners, respectively. Inthis example, the areas 520A, 520B extend beyond the edges of theroadway defined by the fog lines 412, 414.

As another instance, in construction zones, construction workers areseen to walk across the road even in the absence of mapped crosswalks.As such, the area for the unmarked crosswalk may be identified tocorrespond to an entire area of the construction zone or an area that issome predetermined distance larger than the area of the constructionzone. Such areas of construction zones may be identified as a polygon byanother detector of the vehicle which identifies areas for constructionzones. Sometimes, cones or pylons are placed to indicate theconstruction area. In such instances, the area may be identified bydrawing a polygon around the region cordoned off by the cones or pylons,and extending it across the road, and even some small distance beyondthe shoulder area in some instances.

Returning to the example of FIG. 6A, as shown in FIG. 6B, area 620 maybe determined by drawing a polygon around the construction zone 610 andextending that polygon across the lane 432. In this example, the polygonis slightly larger than the construction zone 610, though the polygonmay be the same size as the construction zone. In this example, the area620 extends beyond the edges of the roadway defined by the fog lines412, 414.

As another instance, where one or more persons, such as police officersor other pedestrian, is directing traffic, pedestrians might be morelikely to cross. This may be especially true near school zones or eventssuch as concerts, sporting events, etc. As such, the area for theunmarked crosswalk may be determined based on a point corresponding tothe location of the pedestrian, for instance, by drawing a rectangle ofa particular size large enough to go across the road around the one ormore persons directing traffic. In situations where a person isdirecting traffic outside of an intersection, the rectangle may have aparticular width of 5 meters or more or less, a length enough to goacross the road, and be centered around the person directing traffic. Insituations where a person directing traffic in an intersection, an areaof an unmarked crosswalk may be a polygon that corresponds to the entirearea of the intersection and/or the aforementioned rectangle of aparticular width centered around the pedestrian whichever is greater.The area of the intersection may be defined in the map information ordetermined by the computing devices 110 from sensor data received fromthe perception system 172.

Returning to the example of FIG. 7A, as shown in FIG. 7B, area 720 maybe determined by drawing a rectangle centered on the person directingtraffic 710 and extending that rectangle across the width of lanes 430,432. Arrows 722, 724 represent the particular width. In this example,the area 720 extends beyond the edges of the roadway defined by the foglines 412, 414. Alternatively, as shown in FIG. 7C, area 720Acorresponds to the entire area of the intersection 402.

As another instance, in an area of a traffic accident, such as where acollision may recently have occurred, pedestrians may be more likely tobe walking around. As such, the area for the unmarked crosswalk may beidentified to correspond to an entire area of an accident zone or anarea that is some predetermined distance larger than the area of theaccident zone. Such accident zones may be identified as a polygon byanother detector of the vehicle which identifies areas for accidentzones or by a remote computing device. For example, a remote assistanceoperator may “draw” a polygon in order to delineate an accident zone andthis polygon may be sent to the vehicle over a network.

Returning to the example of FIG. 8, area 820 may be determined bydrawing a rectangle centered on the traffic accident 810 and extendingthat rectangle across the width of lane 430. In this example, the area820 extends beyond the edges of the roadway defined by the fog lines 410(though this is a consequence of the area of the traffic accident 810),412.

As another instance, in areas where there is increased emergency vehicleactivity, such as a group of police or fire trucks gathered together,pedestrians may be more likely to be walking around. As such, the areafor the unmarked crosswalk may be identified to include all of theidentified emergency vehicles plus some additional buffer area ordistance, such as an additional 3-meter distance or more or less.Alternatively, distinct areas corresponding to a plurality of unmarkedcrosswalks may be identified as with the school bus examples above. Insome instances, these areas may be connected to one another in order toidentify a larger area for the unmarked crosswalk.

Returning to the example of FIG. 9A, as shown in FIG. 9B, an area 920may be determined by drawing a rectangle centered on the emergencyvehicle 910 and including some buffer area around the emergency vehicle,and extending that rectangle across the width of lanes 430, 432. Arrows922, 924 represent the additional buffer distance. In this example, thearea 920 extends beyond the edges of the roadway defined by the foglines 410 (though this is a consequence of the area of the emergencyvehicle 910), 412.

As another instance, in residential areas, larger parking lots, andother low-speed areas, pedestrians often cross the road even in theabsence of mapped or painted crosswalks, especially in situations inwhich a sidewalk continues to the edge of the road (e.g. There is asidewalk curb cut). As such, the area for the unmarked crosswalk may beidentified as the entirety of such areas. Returning to the example ofFIG. 10, area 1020 may be determined to be the same as residential area1010.

Returning to FIG. 3, at block 350, the vehicle is controlled in theautonomous driving mode based on the determination and the area. Forinstance, once an area for an unmarked crosswalk situation isdetermined, such as areas 520, 520A, 520B, 620, 720, 720A, 820, 920,1020, certain behaviors for responding to crosswalks may be engaged,activated, or otherwise “turned on” in order to enable the vehicle 100to react to the unmarked crosswalk as if it were an actual crosswalk. Inother words, these behaviors may be used by the computing devices 110 tocontrol the vehicle 100 as the vehicle approaches and passes through theareas 520, 520A, 520B, 620, 720, 720A, 820, 920, 1020.

One example behavior may include giving pedestrians precedence such thatthe vehicle 100 will react more strongly and is more likely to yield topedestrians within or near (outside of) the area of the unmarkedcrosswalk. In addition, computing devices 110 may then cause the vehicle100, for instance by changing parameters of the planning system 168, tomaintain larger time and space buffers to pedestrians (as compared tosituations in which a pedestrian is jaywalking or waiting near an edgeof the road without intending to cross or walk along the edge of theroad) while making trajectory determinations (e.g. geometry and speed).Another example behavior may include yielding to pedestrians waiting tocross a roadway such that the vehicle 100 may be more likely to stop fora pedestrian waiting on a side of the road to cross at or near the areaof the unmarked crosswalk. Another example behavior may includeocclusion reasoning. For example, the computing devices 110 may turn onocclusion reasoning such that the computing devices 110 may reason aboutoccluded pedestrians who could enter and/or traverse the area of theunmarked crosswalk. The computing devices 110 may then cause the vehicle100 to react appropriately to these occluded pedestrians. Anotherexample behavior may include enabling invitation yielding. For example,the computing devices 110 may cause the vehicle 100 to be more likely toyield if the vehicle perception system 172 observes another vehicleyielding to some object within or near (outside of) the area of theunmarked crosswalk. Another example behavior may include allowing thevehicle 100 to spend more resources detecting, predicting, computingwhile approaching and passing through the identified area. This mayinclude putting more computing power into analyzing images forpedestrians or even identifying objects as pedestrians even if otherwisethe confidence is such identifications would be too low to do so.

The features described herein may enable an autonomous vehicle toidentify situations in which pedestrians may be likely to be crossing,attempting to cross, or simply walking in a road. For instance, incertain situations such as near a school bus or school zone,pedestrians, including children, could come from occluded locations andotherwise be unexpected. In constructions zones, construction works maycross a roadway at various locations. When persons are directingtraffic, pedestrians may be more likely to cross the road outside of acrosswalk, and especially at events such as concerts or sporting events.In areas where a collision may have recently occurred or where there isincreased emergency vehicle activity (such as a group of fire trucks,ambulances or police vehicles), pedestrians may be more likely to crossa roadway. Also in residential or other low-speed areas, pedestrians mayalso tend to cross roadways and even intersection in the absence ofcrosswalks. The features described herein may allow an autonomousvehicle to automatically engage certain behaviors useful for crosswalksin instances where such behaviors would be safer, more helpful, andappropriate even if not legally required. Overall, the featuresdescribed herein may greatly improve safety in situations wherepedestrians could be or are crossing a road outside of a crosswalk.

Unless otherwise stated, the foregoing alternative examples are notmutually exclusive, but may be implemented in various combinations toachieve unique advantages. As these and other variations andcombinations of the features discussed above can be utilized withoutdeparting from the subject matter defined by the claims, the foregoingdescription of the embodiments should be taken by way of illustrationrather than by way of limitation of the subject matter defined by theclaims. In addition, the provision of the examples described herein, aswell as clauses phrased as “such as,” “including” and the like, shouldnot be interpreted as limiting the subject matter of the claims to thespecific examples; rather, the examples are intended to illustrate onlyone of many possible embodiments. Further, the same reference numbers indifferent drawings can identify the same or similar elements.

The invention claimed is:
 1. A method for controlling a vehicle havingan autonomous driving mode, the method comprising: receiving, by one ormore processors of the vehicle, sensor data generated by a perceptionsystem of a vehicle, the sensor data identifying objects in anenvironment of the vehicle; receiving, by the one or more processors ofthe vehicle, a trajectory corresponding to a future path of the vehicle;determining, by the one or more processors of the vehicle, that thevehicle is approaching an unmarked crosswalk situation based on thetrajectory and the sensor data; in response to the determination,identifying, by the one or more processors of the vehicle, an areacorresponding to an unmarked crosswalk based on a type of the unmarkedcrosswalk situation; and controlling, by the one or more processors ofthe vehicle, the vehicle in the autonomous driving mode based on thedetermination and the area.
 2. The method of claim 1, wherein theunmarked crosswalk situation corresponds to a situation in which thereis no marked crosswalk in the environment.
 3. The method of claim 1,wherein the unmarked crosswalk situation corresponds to a situation inwhich there is no marked crosswalk in any pre-stored map informationused to control the vehicle in the autonomous driving mode.
 4. Themethod of claim 1, wherein determining that the vehicle is approachingan unmarked crosswalk situation includes determining whether theenvironment includes a school zone.
 5. The method of claim 1, whereindetermining that the vehicle is approaching an unmarked crosswalksituation includes determining whether the environment includes a schoolbus.
 6. The method of claim 5, wherein identifying the area is based ona location of the school bus.
 7. The method of claim 1, whereindetermining that the vehicle is approaching an unmarked crosswalksituation includes determining whether the environment includes aconstruction zone.
 8. The method of claim 7, wherein identifying thearea is based on a polygon of the construction zone.
 9. The method ofclaim 1, wherein determining that the vehicle is approaching an unmarkedcrosswalk situation includes determining whether the environmentincludes one or more persons directing traffic.
 10. The method of claim9, wherein identifying the area is based on a location of the one ormore persons directing traffic.
 11. The method of claim 1, whereindetermining that the vehicle is approaching an unmarked crosswalksituation includes determining whether the environment includes atraffic accident.
 12. The method of claim 11, wherein identifying thearea is based on a polygon for an accident zone of the accident.
 13. Themethod of claim 1, wherein determining that the vehicle is approachingan unmarked crosswalk situation includes determining whether theenvironment includes one or more emergency vehicles.
 14. The method ofclaim 13, wherein identifying the area is based on a location of a firstof the one or more emergency vehicles.
 15. The method of claim 14,further comprising identifying a second area based on a location of asecond of the one or more emergency vehicles, and wherein controllingthe vehicle is further based on the second area.
 16. The method of claim1, wherein controlling the vehicle includes giving a pedestrian in anarea precedence for crossing a road with respect to the vehicle.
 17. Themethod of claim 1, wherein controlling the vehicle includes yielding toa pedestrian on a side of a road outside of the area.
 18. The method ofclaim 1, wherein controlling the vehicle includes using occlusionreasoning for the area in order to predict one or more potentiallyoccluded pedestrians.
 19. The method of claim 1, wherein controlling thevehicle includes allowing the vehicle's computing devices to increaseallocation of computing resources for detecting and responding topedestrians as the vehicle approaches the area.
 20. A vehicle having anautonomous driving mode, the vehicle comprising: a perception system;and a computing device including one or more processors configured to:receive sensor data generated by the perception system, the sensor dataidentifying objects in an environment of the vehicle; receive atrajectory corresponding to a future path of the vehicle; determine thatthe vehicle is approaching an unmarked crosswalk situation based on thereceived trajectory and the sensor data; in response to thedetermination, identify an area corresponding to an unmarked crosswalkbased on a type of the unmarked crosswalk situation; and control thevehicle in the autonomous driving mode based on the determination andthe area.
 21. The vehicle of claim 20, wherein the unmarked crosswalksituation corresponds to a situation in which there is no markedcrosswalk in any pre-stored map information used to control the vehiclein the autonomous driving mode.